Cooking apparatus and control method thereof

ABSTRACT

A cooking apparatus includes a cooking chamber configured to receive food, a camera to photograph inside the cooking chamber, the camera having a lens, a memory configured to store a plurality of cooking control information used to cook the food, and a processor configured to control the camera to photograph the food inside the cooking chamber, wherein the processor is configured to extract state information of the food from the photographed image, and determine a contamination degree of the photographed image according to contamination of the lens of the camera, correct the state information of the food based on the determined contamination degree, acquire first cooking control information corresponding to the corrected state information; and, control the cooking chamber to cook the food according to the acquired cooking control information by acquiring cooking control information.

CROSS-REFERENCE TO RELATED APPLICATION(S)

This application is a continuation application, claiming priority under§365(c), of an

International application No. PCT/KR2022/017193, filed on Nov. 4, 2022,which is based on and claims the benefit of a Korean patent applicationnumber 10-2021-0151685, filed on Nov. 5, 2021, in the KoreanIntellectual Property Office, the disclosure of which is incorporated byreference herein in its entirety.

BACKGROUND 1. Field

The disclosure relates to a cooking apparatus and a control methodthereof. More particularly, the disclosure relates to a cookingapparatus for acquiring state information of food a photographed imageof food inside a cooking chamber and cooking food according to the stateinformation of the food, and a control method thereof

2. Description of Related Art

A cooking apparatus is an apparatus capable of performing variouscooking functions, such as baking, frying, boiling, or the like.Recently, a cooking apparatus equipped with a camera in the cookingapparatus to photograph food to acquire information about the food hasbeen developed.

However, the prior cooking apparatus has a problem in that when food isphotographed in a state where a camera lens is contaminated due to steamor foreign substances generated inside the cooking chamber, informationabout the food from the photographed image is unreliable.

SUMMARY

The disclosure is to solve the problem described above, and an object ofthe disclosure is to provide a cooking apparatus for performingautomatic cooking by correcting state information of food extracted froma photographed image according to a degree of contamination of a cameralens, and a control method thereof.

In addition, an embodiment of acquiring state information of food usingsensing information other than the photographed image is included whenreliable state information cannot be acquired from the photographedimage since the lens is excessively contaminated.

In order to achieve the above object, according to an embodiment of thedisclosure, there is provided a cooking apparatus comprising a cookingchamber configured to receive food, a camera to photograph inside thecooking chamber, the camera having a lens, a memory configured to storea plurality of cooking control information used to cook the food and, aprocessor configured to control the camera to photograph the food insidethe cooking chamber. The processor is further configured to extractstate information of the food from the photographed image, and determinea contamination degree of the photographed image according tocontamination of the lens of the camera, correct the state informationof the food based on the determined contamination degree, acquirecooking first control information corresponding to the corrected stateinformation, and control the cooking chamber to cook the food accordingto the acquired first cooking control information.

The processor is further configured to, based on the determinedcontamination degree being equal to or greater than a first threshold,acquire second cooking control information corresponding to a resultsensed by at least one another sensor stored in the memory, and controlthe cooking chamber to cook the food according to the acquired secondcooking control information.

The processor is further configured to, based on the determinedcontamination degree being less than the first threshold value andgreater than or equal to a second threshold value, acquire cookingcontrol information corresponding to state information corrected basedon the contamination degree from the memory, and based on thecontamination degree being less than the second threshold value, acquirecooking control information corresponding to uncorrected stateinformation from the memory.

The apparatus may further comprises a temperature sensor and a timer,wherein the processor is further configured to, based on thecontamination degree being equal to or greater than the first thresholdvalue, estimate the state information of the food based on a temperaturechange sensed by the temperature sensor and a time counted by the timer,and acquire cooking control information corresponding to the estimatedstate information from the memory.

The processor is further configured to, determine the contaminationdegree by comparing a size of a region of interest in which the food isidentified in the photographed image and a size of a region occluded dueto contamination of the lens of the camera, and correct the stateinformation by adjusting a state value of at least one of a shape, acolor, and a size of the food in proportion to the contamination degree.

The memory is configured to store a plurality of food change patternscorresponding to each of the plurality of cooking control information,and wherein the processor is further configured to acquire a changepattern of the state information by accumulating the state informationof the food for a predetermined time; and search, from the memory, afood change pattern matching the acquired change pattern, and acquirecooking control information corresponding to the searched food changepattern.

The processor is further configured to, based on the food change patternmatching the acquired change pattern being not searched from the memory,search, in the memory, a food change pattern matching the acquiredchange pattern with a variable change pattern.

According to an embodiment of the disclosure, there is provided a methodfor controlling a cooking apparatus, the method comprising photographingfood inside a cooking chamber through a camera of the cooking apparatus,extracting state information of the food from the photographed imagephotographed by the camera, determining a contamination degree of thephotographed image according to contamination of a lens of the camera,correcting the state information of the food based on the determinedcontamination degree, acquiring cooking control informationcorresponding to the corrected state information from the cookingapparatus, the cooking apparatus storing a plurality of cooking controlinformation, and controlling a cooking of the food according to theacquired cooking control information.

The acquiring may include, based on the determined contamination degreebeing equal to or greater than a first threshold, acquiring firstcooking control information corresponding to a result sensed by at leastone another sensor, and the cooking may include cooking the foodaccording to the acquired first cooking control information.

The acquiring may include, based on the determined contamination degreebeing less than the first threshold value and greater than or equal to asecond threshold value, acquiring second cooking control informationcorresponding to state information corrected based on the contaminationdegree and based on the determined contamination degree being less thanthe second threshold value, acquiring third cooking control informationcorresponding to uncorrected state information.

The cooking apparatus may further comprise a temperature sensor and atimer, and the acquiring may include, based on the determinedcontamination degree being equal to or greater than the first thresholdvalue, estimating the state information of the food based on atemperature change sensed by the temperature sensor and a time countedby the timer, and acquiring cooking control information corresponding tothe estimated state information.

The determining the contamination degree may include calculating thecontamination degree by comparing a size of a region of interest inwhich the food is identified in the photographed image and a size of aregion occluded due to contamination of the lens of the camera, and thecorrecting may include correcting the state information by adjusting astate value of at least one of a shape, a color, and a size of the foodin proportion to the contamination degree.

The cooking apparatus is configured to store a plurality of food changepatterns corresponding to each of the plurality of cooking controlinformation, and the acquiring may include acquiring a change pattern ofthe state information by accumulating the state information of the foodfor a predetermined time, and searching a food change pattern matchingthe acquired change pattern and acquiring cooking control informationcorresponding to the searched food change pattern.

The searching may include, based on the food change pattern matching theacquired change pattern being not searched from the cooking apparatus,searching a food change pattern matching the acquired change patternwith a variable change pattern.

According to an embodiment of the disclosure, there is provided Acomputer program product comprising a memory device that comprises oneor more computer executable instructions that when executed by one ormore processors cause the one or more processors of a cooking apparatusto perform a method, the method may comprise photographing food inside acooking chamber through a camera of the cooking apparatus, extractingstate information of the food from the photographed image photographedby the camera, determining a contamination degree of the photographedimage according to contamination of a lens of the camera, correcting thestate information of the food based on the determined contaminationdegree, acquiring cooking control information corresponding to thecorrected state information from the cooking apparatus, the cookingapparatus storing a plurality of cooking control information andcontrolling a cooking of the food according to the acquired cookingcontrol information.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a perspective view schematically illustrating outside andinside of a cooking apparatus according to an embodiment;

FIG. 2 is a view illustrating an image acquired by photographing foodthrough a camera provided in a cooking apparatus according to anembodiment;

FIG. 3 is a block diagram illustrating a configuration of a cookingapparatus according to an embodiment;

FIG. 4 is a block diagram illustrating a configuration of a cookingapparatus according to an embodiment;

FIG. 5 is a flowchart illustrating a method of controlling a cookingapparatus according to an embodiment;

FIG. 6 is a view illustrating an operation of an electronic apparatusaccording to an embodiment;

FIG. 7 is a view illustrating an operation in which a cooking apparatusaccording to an embodiment of the disclosure performs automatic cookingin view of lens contamination;

FIG. 8 is a view illustrating in more detail an operation in which acooking apparatus according to an embodiment of the disclosure performsautomatic cooking in view of lens contamination;

FIG. 9 is a view illustrating an operation of searching for a foodchange pattern matching a change pattern of food state informationacquired from a photographed image from a cooking state information DBstored in a memory;

FIG. 10 is a view illustrating an operation of confirming whether achange pattern of state information of food matches;

FIG. 11 is a view illustrating an operation of varying a change patternof a stored cooking state information DB and confirming whether itmatches a change pattern acquired from a photographed image; and

FIG. 12 is a view illustrating an operation of locally varying a changepattern of a stored cooking state information DB and confirming whetherit matches a change pattern acquired from a photographed image.

DETAILED DESCRIPTION

In the following description, same drawing reference numerals are usedfor the same elements even in different drawings. In the followingdescription, well-known functions or constructions may not be describedin detail if they would obscure the invention in unnecessary detail.Further, dimensions of various elements in the accompanying drawings maybe arbitrarily selected for assisting in a comprehensive understanding.

The terms used in the disclosure and the claims may be general termsselected in view of the functions of the various example embodiments ofthe disclosure. However, such terms may vary depending on an intentionof those skilled in the art, a legal or technical interpretation, anemergence of a new technology, or the like. Also, there may be someterms arbitrarily selected by an applicant. Such terms may be construedaccording to meanings defined in the disclosure, and may also beconstrued based on general contents of the disclosure and a typicaltechnical concept in the art unless the terms are not specificallydefined.

In the description of the disclosure, the order of each step should beunderstood as non-limiting unless a preceding step is to be logicallyand temporally performed before a subsequent step. In other words,except for exceptional cases described above, even if the processdescribed as a subsequent step is performed before the process describedas the preceding step, an essence of the disclosure is not affected, andthe scope of the disclosure should also be defined regardless of theorder of the steps.

The terms “have”, “may have”, “include”, and “may include” used in theexemplary embodiments of the disclosure indicate the presence ofcorresponding features (for example, elements such as numerical values,functions, operations, or parts), and do not preclude the presence ofadditional features.

The terms such as “first,” “second,” and so on may be used to describe avariety of elements, but the elements should not be limited by theseterms. The terms are only used to differentiate one component from othercomponents. For example, the ‘first’ component may be named the ‘second’component, and vice versa, without departing from the scope of thedisclosure.

In the disclosure, components necessary for the description of eachembodiment of the disclosure are described, and thus the disclosure isnot necessarily limited thereto. Accordingly, some components may bechanged or omitted, and the other components may be added. In addition,they may be distributed and arranged in different independent devices.

Various example embodiments of the disclosure will be described ingreater detail below with reference to the accompanying drawings andcontents described in the accompanying drawings, but the disclosure isnot restricted or limited by the embodiments.

Hereinafter, various example embodiments will now be explained in detailwith reference to the accompanying drawings.

FIG. 1 is a perspective view schematically illustrating outside andinside of a cooking apparatus according to an embodiment.

Referring to FIG. 1 , the cooking apparatus 100 may include a main body10 forming an exterior, a cooking chamber 110 for accommodating food, adoor 11 for opening and closing the cooking chamber 110, and a camera120 for photographing the inside of the cooking chamber 110. The cookingapparatus 100 may be implemented as various devices that perform acooking operation, such as an oven, a microwave oven, an air fryer, orthe like.

The camera 120 may acquire a photographed image by photographing foodlocated in the cooking chamber 110 while the food is being cooked in thecooking chamber 110. In that configuration, the camera 120 may be placedinside the cooking chamber 110 to photograph a surface of the food whilethe food is being cooked.

Meanwhile, a lens exposed in a direction of the cooking chamber 110 fromthe camera 120 may be included, and a surface of the lens may becontaminated by foreign substances such as vapor (e.g., vapor, gas . . .) or oil splattered from the food, generated during a cooking process.If the lens is contaminated, the photographed image photographed by thecamera 120 may be affected.

FIG. 2 is a view illustrating an image acquired by photographing foodthrough a camera provided in a cooking apparatus according to anembodiment.

Referring to FIG. 2 , a difference between a food photographed image 210in a state in which a lens is not contaminated and food photographedimages 221 and 222 in a state in which the lens is contaminated may beidentified. In the image photographed while the lens is contaminated, ashape of the food may appear blurry 221 or a part of the food may beoccluded 222.

The cooking apparatus 100 may extract state information of the food fromthe photographed image, and state information extracted from the imageacquired by photographing the food while the lens is contaminated maynot properly reflect an actual state of the food.

Accordingly, the cooking apparatus 100 according to an embodiment of thedisclosure may correct state information of food according to a lenscontamination degree, and propose a method of cooking using thecorrected state information.

Meanwhile, features of the cooking apparatus 100 according to anembodiment of the disclosure are not limited to the case where the lensconfiguration of the camera 120 is contaminated, and may be applied tovarious cases that may affect the photographed image, such as a casethat the other components (e.g., transparent screen) placed on a regionbetween the camera 120 and food are contaminated.

FIG. 3 is a block diagram illustrating a configuration of a cookingapparatus according to an embodiment.

Referring to FIG. 3 , the cooking apparatus 100 may include a cookingchamber 110, a camera 120, a memory 130, and a processor 140.

The cooking chamber 110 may include a space for accommodating food andis configured to perform a cooking operation on the food. The cookingchamber 110 may include a heating member for generating heat, a drivingdevice capable of rotating and moving food, or the like.

The camera 120 may be provided inside the cooking chamber 110, and mayphotograph inside of the cooking chamber 110. Particularly, the camera120 may photograph food accommodated in the cooking chamber 110.

Various instructions, programs, or data required for the operation ofthe cooking apparatus 100 or the processor 140 may be stored in thememory 130.

A plurality of cooking control information may be stored in the memory130. Information on various food that can be cooked by the cookingapparatus 100 may be stored in the memory 130, and information on acooking method for each food may be stored. For example, the cookingcontrol method stored in the memory 130 may include information such asa cooking time, a cooking temperature, a cooking pattern, or the like.

The processor 140 may be electrically connected to the cooking chamber110, the camera 120, and the memory 130 to control the overall operationof the cooking apparatus 100.

The processor 140 may control the camera 120 to photograph the foodlocated inside the cooking chamber 110. The processor 140 may controlthe cooking chamber 110 to extract state information of food from thephotographed image photographed by the camera 120, calculate acontamination degree of the photographed image according to lenscontamination of the camera 120, correct the state information of thefood based on the contamination degree, and acquire cooking controlinformation corresponding to the corrected state information from thememory 130 to cook food according to the cooking control information.

FIG. 4 is a block diagram illustrating a configuration of a cookingapparatus according to an embodiment.

Referring to FIG. 4 , the cooking apparatus 100 may include the cookingchamber 110, the camera 120, the memory 130, the processor 140, thesensor 150, the communicator 160, and an input/output interface 170. Adetailed description of constitutional elements overlapping with theconfiguration illustrated in FIG. 3 among the configuration illustratedin FIG. 4 will be omitted.

The sensor 150 may be located in the cooking chamber 110 and sensecooking-related information while food is being photographed. Theinformation related to cooking may include a temperature of the surfaceof the food located in the cooking chamber 110, a temperature inside thefood, a moisture content of the surface of the food, a moisture contentin the food, a temperature inside the cooking apparatus 100, internalhumidity of the cooking apparatus 100, or the like. Accordingly, thesensor 150 may include a temperature probe sensor capable of measuringthe internal temperature of food, a non-contact temperature sensorcapable of measuring the surface temperature of food, a temperaturesensor capable of sensing the internal temperature of the cookingapparatus 100, and a humidity sensor for sensing a change in the amountof water vapor inside the cooking apparatus 100. According to anotherembodiment, the sensor may be implemented as another sensor such as aweight sensor for sensing a weight of food located in the cookingapparatus 100 or a smoke sensor for sensing smoke generated in thecooking apparatus 100.

When a contamination degree of the lens of the camera 120 is equal to orgreater than a first threshold, the processor 140 may acquire, from thememory 130, cooking control information corresponding to a result sensedby at least one other sensor 150 other than the camera 120. Accordingly,when the lens of the camera 120 is excessively contaminated and thusreliable state information of the food cannot be obtained from thephotographed image, by using a sensing result of the sensor 150 ratherthan the photographed image, it may compensate for the disadvantage thatthe state information of food acquired through the photographed image isunreliable due to lens contamination.

The communicator 160 may transmit and receive various information byperforming communication with an external device. For this operation,the communicator 160 may communicate with an external device through alocal area network (LAN), an Internet network, and a mobilecommunication network, and may also communicate with an external devicethrough various communication methods such as Bluetooth (BT), BluetoothLow Energy (BLE), Wireless fidelity Direct (WI-FI Direct), Zigbee, NFC,or the like. For this operation, the communicator 160 may includevarious communication modules for performing network communication. Forexample, the communicator 160 may include a Bluetooth chip, a Wi-Fichip, a wireless communication chip, or the like.

The input/output interface 170 may be configured to receive various usercommands or provide various information to the user. For example, thecooking apparatus 100 may receive a user command for controlling thecooking apparatus 100 through the input/output interface 170 or receivea user command for transmitting to an external device. For example, amicrophone for receiving a user's spoken voice, a display for outputtinginformation in a visual form, a speaker for outputting information in anauditory form, or the like may be included.

FIG. 5 is a flowchart illustrating a method of controlling a cookingapparatus according to an embodiment.

Referring to FIG. 5 , a method of controlling a cooking apparatusaccording to an embodiment of the disclosure includes photographing foodlocated inside the cooking chamber 110 through the camera 120 (S510),extracting state information of the food from the photographed image(S520), calculating a contamination degree of the photographed imageaccording to lens contamination of the camera 120 (S530), correcting thestate information of the food based on the contamination degree (S540),acquiring cooking control information corresponding to the correctedstate information from a cooking apparatus storing a plurality ofcooking control information (S550) and cooking food according to theacquired cooking control information (S560).

The cooking apparatus 100 may photograph the food located inside thecooking chamber 110 through the camera 120 (S510).

In addition, the cooking apparatus 100 may extract state information ofthe food from the photographed image photographed by the camera 120(S520).

The cooking apparatus 100 may calculate the contamination degree of thephotographed image according to the lens contamination of the camera 120(S530).

In that configuration, the cooking apparatus 100 may calculate thecontamination degree by comparing a size of a region of interest inwhich the food is identified in the photographed image and a size of aregion occluded due to lens contamination of the camera.

The cooking apparatus 100 may correct the state information of the foodbased on the contamination degree (S540).

In that configuration, the cooking apparatus 100 may correct the stateinformation by adjusting a state value of at least one of shape, color,and size of the food in proportion to the contamination degree.

The cooking apparatus 100 may acquire cooking control informationcorresponding to the corrected state information from the cookingapparatus storing a plurality of cooking control information (S550).

In that configuration, when the contamination degree is equal to orgreater than the first threshold, cooking control informationcorresponding to a result sensed by at least one other sensor may beacquired. The cooking apparatus 100 may estimate the state informationof the food based on a temperature change sensed by the temperaturesensor and a time counted by a timer, and acquire cooking controlinformation corresponding to the estimated state information.

Also, when the contamination degree is less than the first thresholdvalue and equal to or greater than a second threshold value, cookingcontrol information corresponding to the state information correctedbased on the contamination degree may be acquired.

Meanwhile, when the contamination degree is less than the secondthreshold, cooking control information corresponding to uncorrectedstate information may be acquired.

In addition, the cooking apparatus 100 may cook food according to theacquired cooking control information (S560).

FIG. 6 is a view illustrating an operation of an electronic apparatusaccording to an embodiment.

When cooking is started, the cooking apparatus 100 may photograph thefood through the camera 120 (S610), and extract a region of interest,which is a region where the food is identified, from the photographedimage (S620).

The cooking apparatus 100 may extract state information of the food fromthe photographed image photographed by the camera 120 (S630) andidentify lens contamination degree of the camera 120 (S640).

The cooking apparatus 100 may extract state information of the food fromthe region of interest of the photographed image (S630). The cookingapparatus 100 may acquire state values such as a shape, color, and sizeof the food. For example, the cooking apparatus 100 may acquire a sizevalue of the food by identifying a region occupied by the food on thephotographed image by extracting a boundary line of the food from thephotographed image, and acquire color values of pixels corresponding tothe region of interest of the photographed image in relation to a colorof the food.

The extracted food state information may be matched with informationincluded in a cooking state information database DB stored in the memory130. For example, a plurality of food change patterns corresponding toeach of a plurality of cooking control information may be stored in thememory 130, and the processor 140 may acquire the change pattern of thestate information by accumulating the state information of the food fora predetermined time, and search a food change pattern matching theacquired change pattern from the memory 130 to acquire cooking controlinformation corresponding to the searched food change pattern from thememory 130. A detailed description related thereto will be describedlater with reference to FIGS. 9 to 12 .

The cooking apparatus 100 may identify a lens contamination degree bycalculating a contamination degree according to the lens contaminationof the camera 120 from the photographed image (S640). The cookingapparatus 100 may identify the degree to which the lens of the camera120 is occluded by vapor (e.g., vapor, gas . . . ) by extractinggradients of adjacent pixels on the photographed image. Also, thecooking apparatus 100 may identify a region where the lens of the camera120 is occluded by a foreign substance by extracting a boundary line orsensing a change point on the photographed image.

The cooking apparatus 100 may calculate the contamination degree invarious ways. For example, the cooking apparatus 100 may calculate thecontamination degree by comparing a size of a region of interest inwhich the food is identified in the photographed image and a size of aregion occluded due to lens contamination of the camera 120. In thatconfiguration, the contamination degree may be calculated as a ratio ofthe size of the occluded region to the size of the region of interest.According to another embodiment, the cooking apparatus 100 may calculatethe ratio of the size of the region occluded by the lens contaminationof the camera 120 to an overall size of the photographed image, as thecontamination degree.

Meanwhile, the cooking apparatus 100 may correct the state informationof the food based on the contamination degree (S650). In thatconfiguration, the cooking apparatus 100 may adjust the state valueincluded in the food state information based on the contaminationdegree. For example, the cooking apparatus 100 may adjust a state valueof at least one of a shape, a color, and a size of the food inproportion to the degree of contamination. When the contamination degreeis the ratio of the size of the region occluded by a foreign substanceto the overall size of the photographed image, the cooking apparatus 100may adjust the size value of the food to increase in proportion to thecontamination degree value. For example, when the region occluded byforeign substances is 40% of a total size of the photographed image, thesize of the food may be adjusted to increase by 40% by estimating thatthe size of the food is occluded by 40%. Accordingly, the stateinformation of the food based on the contamination degree may becorrected.

In addition, the cooking apparatus 100 may perform automatic cooking inview of lens contamination (S660). A detailed description relatedthereto will be described later with reference to FIGS. 7 to 8 .

FIG. 7 is a view illustrating an operation in which a cooking apparatusaccording to an embodiment of the disclosure performs automatic cookingin view of lens contamination.

Referring to FIG. 7 , the cooking apparatus 100 may perform a cookingoperation by applying different cooking algorithms according to thecontamination degree of the lens of the camera 120.

When the contamination degree is less than the second threshold value(S710, N), the cooking apparatus 100 may apply the automatic cookingalgorithm by identifying the lens of the camera 120 is not contaminated(S711). The automatic cooking algorithm may refer to an algorithm forcontrolling the cooking apparatus 100 according to cooking controlinformation pre-stored in the memory 130.

When the contamination degree of the lens of the camera 120 is less thanthe first threshold value and greater than or equal to the secondthreshold value (S720, N), the cooking apparatus 100 may acquire cookingcontrol information corresponding to the state information correctedbased on the contamination degree from the memory 130, and may apply acompensation algorithm for controlling the cooking apparatus 100 toperform a cooking operation according to the acquired cooking controlinformation (S721).

Meanwhile, when the contamination degree is equal to or greater than thefirst threshold value (S720, Y), cooking control informationcorresponding to a result sensed by at least one other sensor may beacquired from the memory 130, and an alternative algorithm forcontrolling the cooking apparatus 10 may be applied to perform a cookingoperation according to the acquired cooking control information (S730).

FIG. 8 is a view illustrating in more detail an operation in which acooking apparatus according to an embodiment of the disclosure performsautomatic cooking in view of lens contamination.

Referring to FIG. 8 , when the contamination degree is less than thesecond threshold value (S810, N), which means that when it is identifiedthat the lens of the camera 120 is not contaminated, the cookingapparatus 100 may identify whether an automatic cooking algorithm isbeing applied to the current cooking apparatus 100 (S820). When it isidentified that the current automatic cooking algorithm is applied(S820, Y), the cooking apparatus 100 may maintain the current state ofapplying the automatic cooking algorithm (S821), and when an algorithmother than the automatic cooking algorithm is applied (S820, N), thecooking apparatus 100 may change the mode to the automatic cookingalgorithm (S822).

Meanwhile, when the contamination degree is greater than or equal to thesecond threshold value (S810, Y), which means that when it is identifiedthat the lens of the camera 120 is contaminated, the cooking apparatus100 may identify whether the contamination degree is greater than orequal to the first threshold value greater than the second thresholdvalue (S830).

When the contamination degree is equal to or greater than the firstthreshold value, it may mean that the lens of the camera 120 isexcessively contaminated, and thus reliable state information of thefood cannot be acquired from the photographed image. Accordingly, whenthe contamination degree is equal to or greater than the first thresholdvalue (S830, Y), cooking control information corresponding to a resultsensed by at least one other sensor 150 may be acquired from a memory132 for storing the second cooking state information DB, and analternative algorithm for controlling the cooking apparatus 10 may beapplied to perform a cooking operation according to the acquired cookingcontrol information (S870).

Meanwhile, when the contamination degree is less than the firstthreshold value and greater than or equal to the second threshold value(S830, N), the cooking apparatus 100 may correct the state informationbased on the contamination degree and acquire the cooking controlinformation corresponding to the corrected state information from thefirst cooking state information DB stored in a memory 131. In thatconfiguration, whether or not a cooking pattern can be predicted may beidentified (S840) according to whether or not cooking controlinformation matching the corrected state information is searched fromthe memory 131. A detailed description related to a search operation forcooking control information matching the state information will bedescribed below with reference to FIGS. 9 to 12 .

When it is identified that the cooking pattern is predictable from thestate information of the food extracted from the photographed imagephotographed by the camera 120 (S840, Y), the cooking apparatus 100 mayperform the cooking operation by applying a cooking pattern predictionalgorithm (S860). In that configuration, the cooking pattern predictionalgorithm may search, from the memory 131, for a food change patternmatching the change pattern of the state information of the food, andperform the cooking operation according to the cooking controlinformation corresponding to the searched food change pattern.

Meanwhile, if it is identified that the cooking pattern is unpredictable(S840, N), the cooking apparatus 100 may perform a cooking operation byapplying a proportional compensation algorithm (S850). In thatconfiguration, the proportional compensation algorithm may correct thestate information (e.g., state values related to shape, color, and size)of the food in proportion to the contamination degree, and perform acook operation by acquiring cooking control information corresponding tothe corrected state information from the memory 131.

Accordingly, a cooking algorithm may be applied in different waysaccording to a reliability of the food state information that can beextracted from the photographed image.

FIG. 9 is a view illustrating an operation of searching for a foodchange pattern matching a change pattern of food state informationacquired from a photographed image from a cooking state information DBstored in a memory.

Referring to FIG. 9 , the processor 140 may acquire a change pattern 910of state information by accumulating state information of food for apredetermined time, and search, from the memory 130, a food changepattern 920 matching the acquired change pattern 910. When the changepattern 920 matching the acquired change pattern 910 is searched, acooking operation may be performed by acquiring cooking controlinformation corresponding to the searched food change pattern 920 fromthe memory 130.

Accordingly, a change in the cooking state may be predicted by analyzingthe state information of the food, and cooking suitable for the food maybe automatically performed by performing a cooking operation accordingto the predicted result.

Meanwhile, searching for a matching change pattern from the food changepattern stored in the memory 130 may be failed.

FIG. 10 is a view illustrating an operation of confirming whether achange pattern of state information of food matches.

Referring to FIG. 10 , there may be a change pattern 1010 of stateinformation that matches the food change pattern of the cooking stateinformation DB stored in the memory 130, but there may be a changepattern of state information that does not search a matching changepattern 1020. In that configuration, the processor 140 may search for amatching change pattern by varying the change pattern acquired from thephotographed image or the change pattern stored in the memory 130.

FIG. 11 is a view illustrating an operation of varying a change patternof a stored cooking state information DB and confirming whether itmatches a change pattern acquired from a photographed image.

When the processor 140 fails to search, from the memory 130, for a foodchange pattern matching the acquired change pattern, the processor 140may acquire a matching change pattern by matching the acquired changepattern with a change pattern 1120 varying the change pattern 1110stored in the memory 130.

Referring to FIG. 11 , the variable change pattern 1120 and the changepattern acquired from the photographed image may be matched by varyingthe overall scale of the stored change pattern 1110.

Meanwhile, referring to FIG. 12 , the variable change pattern 1220 andthe change pattern acquired from the photographed image may be matchedby locally varying a scale of the stored change pattern 1210.

Meanwhile, when the processor 140 fails to search, from the memory 130,a food change pattern matching the acquired change pattern, theprocessor 140 may acquire a matching change pattern by searching, fromthe memory 130, a food change pattern matching the acquired changepattern with the variable change pattern.

According to various embodiments described above, computer instructionsfor performing processing operations of the cooking apparatus accordingto the various embodiments described above may be stored in anon-transitory computer-readable medium. The computer instructionsstored in the non-transitory computer-readable medium may cause aparticular device to perform processing operations on the deviceaccording to the various embodiments described above when executed bythe processor of the particular device.

The non-transitory computer readable recording medium refers to a mediumthat stores data and that can be read by devices. For example, theabove-described various applications or programs may be stored in thenon-transitory computer readable medium, for example, a compact disc(CD), a digital versatile disc (DVD), a hard disc, a Blu-ray disc, auniversal serial bus (USB), a memory card, a read only memory (ROM), orthe like, and may be provided.

The methods according to various embodiments described above may beprovided as a part of a computer program product. The computer programproduct may be traded between a seller and a buyer. The computer programproduct may be distributed in a form of the machine-readable storagemedia (e.g., compact disc read only memory (CD-ROM) or distributedonline through an application store (e.g., PlayStore TM). In a case ofthe online distribution, at least a portion of the computer programproduct may be at least temporarily stored or provisionally generated onthe storage media such as a manufacturer's server, the applicationstore's server, or a memory in a relay server.

While the disclosure has been illustrated and described with referenceto various example embodiments, it will be understood that the variousexample embodiments are intended to be illustrative, not limiting. Itwill be further understood by those skilled in the art that variouschanges in form and detail may be made without departing from the truespirit and full scope of the disclosure, including the appended claimsand their equivalents.

What is claimed is:
 1. A cooking apparatus comprising: a cooking chamberconfigured to receive food; a camera to photograph inside the cookingchamber, the camera having a lens; a memory configured to store aplurality of cooking control information used to cook the food; and aprocessor configured to control the camera to photograph the food insidethe cooking chamber; wherein the processor is further configured to:extract state information of the food from the photographed image, anddetermine a contamination degree of the photographed image according tocontamination of the lens of the camera, correct the state informationof the food based on the determined contamination degree; acquire firstcooking control information corresponding to the corrected stateinformation; and control the cooking chamber to cook the food accordingto the acquired first cooking control information.
 2. The cookingapparatus of claim 1, wherein the processor is further configured to:based on the determined contamination degree being equal to or greaterthan a first threshold, acquire second cooking control informationcorresponding to a result sensed by at least one another sensor storedin the memory; and control the cooking chamber to cook the foodaccording to the acquired second cooking control information.
 3. Thecooking apparatus of claim 2, wherein the processor is furtherconfigured to: based on the contamination degree being less than thefirst threshold value and greater than or equal to a second thresholdvalue, acquire cooking control information corresponding to stateinformation corrected based on the contamination degree from the memory,and based on the contamination degree being less than the secondthreshold value, acquire cooking control information corresponding touncorrected state information from the memory.
 4. The cooking apparatusof claim 3, further comprising: a temperature sensor; and a timer,wherein the processor is further configured to: based on the determinedcontamination degree being equal to or greater than the first thresholdvalue, estimate the state information of the food based on a temperaturechange sensed by the temperature sensor and a time counted by the timer,and acquire cooking control information corresponding to the estimatedstate information from the memory.
 5. The cooking apparatus of claim 1,wherein the processor is further configured to: determine thecontamination degree by comparing a size of a region of interest inwhich the food is identified in the photographed image and a size of aregion occluded due to contamination of the lens of the camera, andcorrect the state information by adjusting a state value of at least oneof a shape, a color, and a size of the food in proportion to thecontamination degree.
 6. The cooking apparatus of claim 1, wherein thememory is configured to store a plurality of food change patternscorresponding to each of the plurality of cooking control information,and wherein the processor is further configured to: acquire a changepattern of the state information by accumulating the state informationof the food for a predetermined time; and search, from the memory, afood change pattern matching the acquired change pattern, and acquirecooking control information corresponding to the searched food changepattern.
 7. The cooking apparatus of claim 6, wherein the processor isfurther configured to, based on the food change pattern matching theacquired change pattern being not searched from the memory, search, inthe memory, a food change pattern matching the acquired change patternwith a variable change pattern.
 8. A method for controlling a cookingapparatus comprising: photographing food inside a cooking chamberthrough a camera of the cooking apparatus; extracting state informationof the food from the photographed image photographed by the camera;determining a contamination degree of the photographed image accordingto contamination of a lens of the camera; correcting the stateinformation of the food based on the determined contamination degree;acquiring cooking control information corresponding to the correctedstate information from the cooking apparatus, the cooking apparatusstoring a plurality of cooking control information; and controlling acooking of the food according to the acquired cooking controlinformation.
 9. The method of claim 8, wherein the acquiring includes,based on the determined contamination degree being equal to or greaterthan a first threshold, acquiring first cooking control informationcorresponding to a result sensed by at least one another sensor, andwherein the cooking includes cooking the food according to the acquiredfirst cooking control information.
 10. The method of claim 9, whereinthe acquiring includes, based on the determined contamination degreebeing less than the first threshold value and greater than or equal to asecond threshold value, acquiring second cooking control informationcorresponding to state information corrected based on the contaminationdegree, and based on the determined contamination degree being less thanthe second threshold value, acquiring third cooking control informationcorresponding to uncorrected state information.
 11. The method of claim9, wherein the cooking apparatus further comprises: a temperaturesensor; and a timer, wherein the acquiring includes, based on thedetermined contamination degree being equal to or greater than the firstthreshold value, estimating the state information of the food based on atemperature change sensed by the temperature sensor and a time countedby the timer, and acquiring cooking control information corresponding tothe estimated state information.
 12. The method of claim 8, wherein thedetermining the contamination degree includes calculating thecontamination degree by comparing a size of a region of interest inwhich the food is identified in the photographed image and a size of aregion occluded due to contamination of the lens of the camera, andwherein the correcting includes correcting the state information byadjusting a state value of at least one of a shape, a color, and a sizeof the food in proportion to the contamination degree.
 13. The method ofclaim 8, wherein the cooking apparatus is configured to store aplurality of food change patterns corresponding to each of the pluralityof cooking control information, and wherein the acquiring includes:acquiring a change pattern of the state information by accumulating thestate information of the food for a predetermined time, and searching afood change pattern matching the acquired change pattern; and acquiringcooking control information corresponding to the searched food changepattern.
 14. The method of claim 13, wherein the searching includes,based on the food change pattern matching the acquired change patternbeing not searched from the cooking apparatus, searching a food changepattern matching the acquired change pattern with a variable changepattern.
 15. A computer program product comprising a memory device thatcomprises one or more computer executable instructions that whenexecuted by one or more processors cause the one or more processors of acooking apparatus to perform a method, the method comprising: whereinthe program includes: photographing food inside a cooking chamberthrough a camera of the cooking apparatus; extracting state informationof the food from the photographed image photographed by the camera;determining a contamination degree of the photographed image accordingto contamination of a lens of the camera; correcting the stateinformation of the food based on the determined contamination degree;acquiring cooking control information corresponding to the correctedstate information from the cooking apparatus, the cooking apparatusstoring a plurality of cooking control information; and controlling acooking of the food according to the acquired cooking controlinformation.